Multimedia Computer-based Training And Learning: The Role Of Referential Connections In Supporting Cognitive Learning Outcomes

Multimedia theory has generated a number of principles and guidelines to support computer-based training (CBT) design. However, the cognitive processes responsible for learning, from which these principles and guidelines stem from, are only indirectly derived by focusing on cognitive learning outcome differences. Unfortunately, the effects that cognitive processes have on learning are based on the assumption that cognitive learning outcomes are indicative of certain cognitive processes. Such circular reasoning is what prompted this dissertation. Specifically, this dissertation looked at the notion of referential connections, which is a prevalent cognitive process that is thought to support knowledge acquisition in a multimedia CBT environment. Referential connections, and the related cognitive mechanisms supporting them, are responsible for creating associations between verbal and visual information; as a result, their impact on multimedia learning is theorized to be far reaching. Therefore, one of the main goals of this dissertation was to address the issue of indirectly assessing cognitive processes by directly measuring referential connections to (a) verify the presence of referential connections, and (b) to measure the extent to which referential connections affect cognitive learning outcomes. To achieve this goal, a complete review of the prevalent multimedia theories was brought fourth. The most important factors thought to be influencing referential connections were extracted and cataloged into variables that were manipulated, fixed, covaried, or randomized to empirically examine the link between referential connections and learning. Specifically, this dissertation manipulated referential connections by varying the temporal presentation of modalities and the color coding of instructional material. Manipulating the temporal presentation of modalities was achieved by either presenting modalities simultaneously or sequentially. Color coding manipulations capitalized on pre-attentive highlighting and pairing of elements (i.e., pairing text with corresponding visuals). As such, the computer-based training varied color coding on three levels: absence of color coding, color coding without pairing text and corresponding visual aids, and color coding that also paired text and corresponding visual aids. The modalities employed in the experiment were written text and static visual aids, and the computer-based training taught the principles of flight to naive participants. Furthermore, verbal and spatial aptitudes were used as covariates, as they consistently showed to affect learning. Overall, the manipulations were hypothesized to differentially affect referential connections and cognitive learning outcomes, thereby altering cognitive learning outcomes. Specifically, training with simultaneously presented modalities was hypothesized to be superior, in terms of referential connections and learning performance, to a successive presentation, and color coding modalities with pairing of verbal and visual correspondents was hypothesized to be superior to other forms of color coding. Finally, it was also hypothesized that referential connections would positively correlate with cognitive learning outcomes and, indeed, mediate the effects of temporal contiguity and color coding on learning. A total of 96 were randomly assigned to one of the six experimental groups, and were trained on the principles of flight. The key construct of referential connections was successfully measured with three methods. Cognitive learning outcomes were captured by a traditional declarative test and by two integrative (i.e., knowledge application) tests. Results showed that the two multimedia manipulation impacted cognitive learning outcomes and did so through corresponding changes of related referential connections (i.e., through mediation). Specifically, as predicted, referential connections mediated the impact of both temporal contiguity and color coding on lower- and higher-level cognitive learning outcomes. Theoretical and practical implications of the results are discussed in relation to computer-based training design principles and guidelines. Specifically, theoretical implications focus on the contribution that referential connections have on multimedia learning theory, and practical implications are brought forth in terms of instructional design issues. Future research considerations are described as they relate to further exploring the role of referential connections within multimedia CBT paradigms

Automatic Gaze Classification for Aviators: Using Multi-task Convolutional Networks as a Proxy for Flight Instructor Observation

In this work, we investigate how flight instructors observe aviator scan patterns and assign quality to an aviator\u27s gaze. We first establish the reliability of instructors to assign similar quality to an aviator\u27s scan patterns, and then investigate methods to automate this quality using machine learning. In particular, we focus on the classification of gaze for aviators in a mixed-reality flight simulation. We create and evaluate two machine learning models for classifying gaze quality of aviators: a task-agnostic model and a multi-task model. Both models use deep convolutional neural networks to classify the quality of pilot gaze patterns for 40 pilots, operators, and novices, as compared to visual inspection by three experienced flight instructors. Our multi-task model can automate the process of gaze inspection with an average accuracy of over 93.0% for three separate flight tasks. Our approach could assist existing flight instructors to provide feedback to learners, or it could open the door to more automated feedback for pilots learning to carry out different maneuvers

Supporting Decision-Making in the Battlefield: Utility of Multimedia Information Transmission

In the theatre of war, environmental and workload factors (e.g., noise, time pressure) may dramatically overburden operator\u27s cognitive processes. Well designed multimedia technology can successfully minimize both intrinsic (working memory capacity) and extrinsic (inefficient use of information) sources of cognitive load, thereby facilitating the decision-making process. These positive mediating effects of multimedia technology can occur during three distinct phases of human information processing: 1) during sensorial information acquisition, by filtering out unnecessary environmental noise and transferring information via context-dependent modalities; 2) during decision-making, by making critical task-relevant cues more salient; and 3) following response execution, by providing necessary feedback to effectively evaluate the appropriateness of the decision taken. After analyzing the dramatic events that led to many US soldier casualties in a Somali rescue mission, this paper will propose a set of recommendations to help future task forces in urban environments, and to specify the greater goal of multimedia use on the battlefield

Investigating Individual Differences and Instructional Efficiency in Computer-Based Training Environments

This study assessed the extent to which a guided learner-generated questioning strategy could facilitate the acquisition of task-relevant knowledge and improve the instructional efficiency of a computer-based training program for a complex dynamic distributed decision-making task. This study also investigated how individual differences in verbal comprehension ability may interact with this instructional strategy to impact post-training outcomes. Overall, results highlighted the importance of learner aptitudes in complex task training and also showed that the effect of the instructional strategy on knowledge acquisition and the training programa” instructional efficiency was strongest for learners with low verbal comprehension ability. Implications for the design of adaptive learning systems are discusse

Training Individuals For Distributed Teams: Problem Solving Assessment For Distributed Mission Research

In this paper we describe an effort investigating the feasibility and utility of cognitively diagnostic assessment of problem solving when training for distributed team tasks. We utilized computer-based knowledge elicitation methods to assess both relational problem solving, requiring the semantic integration of concepts, and dynamic problem solving, requiring the ability to integrate and apply these concepts. Additionally, we addressed how metacognitive processes interact with learning outcomes when training for complex synthetic task environments. We find first, that multiple methods of assessing problem solving performance are diagnostic of knowledge acquisition for a complex synthetic team task, and second, that general metacomprehension predisposition is related to metacomprehension accuracy in synthetic task environments. © 2002 Elsevier Science Ltd. All rights reserved

Designing Learner-Centered Multimedia Technology

The ubiquitous use of information technology (IT) promotes a fast-paced and dynamic training environment with enormous potential for performance increases in a variety of domains. This reality has many important ramifications, including how best to incorporate multimedia IT into computer-based training (CBT). Well-designed CBT offers us tremendous potential to effectively and efficiently train the workforce, foster learning in academic environments, and improve performance over and above what is currently achieved. Following a learner-centered design approach, in this article, we present an in-depth look at the use of multimedia CBT, as it relates to aptitude-treatment interactions; that is, how various CBT designs can differentially interact with individual learner aptitudes, such as spatial and verbal ability, to influence training outcomes. The goal of this article is to emphasize the importance of learner-centered design when developing multimedia computer-based instructional material for the growing needs of many sectors of society

Training individuals for distributed teams: problem solving assessment for distributed mission research §

In this paper we describe an effort investigating the feasibility and utility of cognitively diagnostic assessment of problem solving when training for distributed team tasks. We utilized computer-based knowledge elicitation methods to assess both relational problem solving, requiring the semantic integration of concepts, and dynamic problem solving, requiring the ability to integrate and apply these concepts. Additionally, we addressed how metacognitive processes interact with learning outcomes when training for complex synthetic task environments. We find first, that multiple methods of assessing problem solving performance are diagnostic of knowledge acquisition for a complex synthetic team task, and second, that general metacomprehension predisposition is related to metacomprehension accuracy in synthetic task environments

Criterion Shift And Anchoring In A Discrimination Training Paradigm: The Importance Of Pre-Training

We report an empirical investigation on pre-training and its effects on overall training accuracy. This study was developed to explore a potential new venue aiming at improving overall training effectiveness for airport X-Ray screeners by means of pretraining. Specifically, we looked at presence or absence of pre-training and how it affected training performance. Pre-training was theoretically framed around the concept of criterion shift and anchoring in a signal detection paradigm. Results showed that pretraining successfully mitigated participants\u27 initial biases brought to the training environment by shifting participant\u27s criterion to a more neutral position in terms of discrimination training rules and heuristics

Understanding Performance And Cognitive Efficiency When Training For X-Ray Security Screening

We describe an experiment designed to understand the X-ray security screener task via investigation of how training environment and content influence perceptual learning. We examined both perceptual discrimination and the presence/absence of clutter during training and how this impacted performance. Overall, the data show that performance was generally better when there were clutter items in the training images. We also examined the diagnosticity of a measure of cognitive efficiency, a combinatory metric that simultaneously considers test performance and workload. In terms of cognitive efficiency, participants who trained in the difficult discrimination with clutter present experienced lower workload during the test relative to their actual performance. The discussion centers on how improved analytical techniques are better able to diagnose the relative effectiveness of training interventions

Computer Based Training And Multimedia Design: The Role Of Spatial Aptitudes In Learning

The present study manipulated both the technology (Microsoft PowerPoint® versus Macromedia Director®) and the design (Text Priming versus No Text Priming) in a computed-based training multimedia environment. The study investigated trainees\u27 spatial aptitude and how it interacted with computer-based training design. The specific computer-based training environment employed in this experiment was based on presenting the principles of flight to naïve participants. A multifaceted knowledge test was developed to assess different levels of knowledge integration. Overall, results suggest that technology type did not play an important role in knowledge acquisition. Priming type also did not show significant effects; however, when factoring trainees\u27 spatial ability, significant aptitude-treatment interactions emerged. We discuss these findings as they relate to computer-based training multimedia paradigms